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Senses of smells are often accompanied by simultaneous visual sensations. Previous studies have 
documented enhanced olfactory performance with concurrent presence of congruent color- or shape- 
related visual cues, and facilitated visual object perception when congruent smells are simultaneously 
present. These visual object-olfaction interactions suggest the existences of couplings between the olfactory 
pathway and the visual ventral processing stream. However, it is not known if olfaction can modulate visual 
motion perception, a function that is related to the visual dorsal stream. We tested this possibility by 
examining the influence of olfactory cues on the perceptions of ambiguous visual motion signals. We 
showed that, after introducing an association between motion directions and olfactory cues, olfaction could 
indeed bias ambiguous visual motion perceptions. Our result that olfaction modulates visual motion 
processing adds to the current knowledge of cross-modal interactions and implies a possible functional 
linkage between the olfactory system and the visual dorsal pathway. 

I n our daily life, we are constantly exposed to multiple sensory inputs from different sensory modalities of 
I varying reliability. Yet we could effortless integrate these parallel sensory signals and maintain a unified 
I perception that allows us to interact with the external world. For example, in a buffet dinner we often use 

not only vision but also olfaction to serve ourselves with the most desirable food. It is well known that visual cues 

such as colors 15 , shapes 6 , pictorial images 7,8 or even abstract symbols 9 could modulate olfactory perceptions. 

However, surprisingly little is known about whether the reverse modulation - olfactory on visual perception - can 

occur 10 . 

Recent studies began to reveal that olfaction can influence visual object perception. In one study 1 subjects were 
asked to search for a visual object embedded in a photograph full of objects. It was shown that the visual object was 
explored faster in the presence of the congruent odor. Similar olfactory modulations on visual object perception 
have also been discovered during binocular rivalry 12 - a well established phenomenon with alternating percep- 
tions of two competing objects that are presented to each eye. The authors have shown that an odorant congruent 
to one of the competing objects prolongs the time that object is visible and shortens its suppression time. These 
olfactory modulations on visual object perception can even occur in the absence of conscious visual awareness 12 
and have lateralization properties, smelling an odor from one nostril modulates the timing of rivalry perceptions 
more for the objects presented in the contralateral visual field relative to those presented in the ipsilateral visual 
field 13 . Those results suggest an early convergence between olfactory system and visual system, specifically the 
visual ventral stream for object representation 14 . Yet, it remains to be tested whether visual motion processing, 
which is associated with the visual dorsal stream, can be modulated by olfactory cues as well. Addressing this 
question will help us better understand interactions between olfactory and visual system. 

It has been shown that the brain integrates multisensory information by weighting each modality in proportion 
to its reliability, compatible with a statistical optimal scheme 15 20 (e.g., Bayesian or maximum-likelihood estima- 
tion 2123 ). In vision- olfaction interactions, since vision undoubtedly prevails olfaction in a normal circumstance, 
such reliability-based multisensory integration theory asserts that vision should modulate olfactory perception 
(as reported in numerous previous studies, for a review see 10 ) while olfaction could hardly influence visual 
perception, unless the reliability of visual information is compromised (e.g., during binocular rivalry 1213 ). For 
visual motion signals, the reliability refers to the levels of directional ambiguity, which is often manipulated by 
varying the relative percentages of coherent moving dots and random moving dots 24,25 . Here, we tested whether 
the perception of ambiguous visual motion direction is influenced by the presence of olfactory cues that are 
associated with directions by classical conditioning pairing 26 . We hypothesized that if there is a interaction 
between visual dorsal pathway and olfactory system, olfactory cues should be able to bias visual motion direction 
perception. On the other hand, if olfactory cues could not modulate visual motion direction perception, this 
would indicate that olfaction-vision interaction might be restricted to the visual ventral stream. 
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Figure 1 | Experimental procedures. During the pairing phase subjects viewed coherently moving dot patches while sniffing the associated banana-like 
and fennel-like odorants sequentially. During the testing phase they indicated the perceived direction of an ambiguous moving dot patch after a third 
sniffing. In experiment 1 the banana-smell was paired with the leftward motion while the fennel-smell was paired with the rightward motion; in 
experiment 2 the reversed pairing was implemented; in experiment 3 both leftward and rightward directions were paired with the same neutral air-smell 
(from purified water). In experiment 4 the motion direction-odorant pairing was randomized from trial to trial. (The pictures of banana and water were 
taken by author S.K. The picture of fennel was taken by author T.Z.) 



Results 

Experiment 1: odors can bias ambiguous visual motion percep- 
tion. First, we would like to see how the perception of ambiguous 
visual motion directions was influenced by the presence of odorant 
stimuli, which paired with motion directions on a trial-by-trial basis. 
Each trial was consisted of two phases: a pairing phase followed by a 
testing phase. Experimental procedures are schematically illustrated 
in figure 1. In experiment 1, we paired leftward moving dot patch 
with the smell of banana and rightward moving dot patch with the 
smell of fennel (Fig. 2A). At the end of each experiment, behavioral 
data was sorted according to testing phase. We found that when 
tested with ambiguous leftward moving dot patch, the overall 
probability of participants perceived rightward motion was 18.9% 
± 3.4% (mean ± SEM) for banana-smell trials, and 35.0% ± 5.6% for 



fennel-smell trials. The difference was statistically significant (t (13) 
= 3.7, p = 0.003). A similar pattern held true for the rightward 
motion perceptions. When tested with ambiguous rightward 
moving dot patch, the probability of perceiving a rightward 
motion was 55.6% ± 5.0% for the banana-smell and 68.5% ± 4.7% 
for the fennel-smell. Again, the percentages of perceiving a rightward 
motion was significant larger in fennel-smell trials as comparing to 
banana-smell trials (t (13) = 3.3, p = 0.005). A two-factor repeated 
measures ANOVA was conducted, including odorants (banana vs. 
fennel) and motion directions (leftward vs. rightward) as factors. 
There were significant effects on odorants (F(l,52) = 9.3, p = 
0.0037) and motion directions (F(l,52) = 54.5, p < 0.001), but no 
significant interactions (F(l,52) = 0.11, p = 0.74). Taken together, 
the results from experiment 1 showed that odorants influenced the 
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Figure 2 | Odorants modulate ambiguous visual motion direction discrimination. (A): When pairing leftward motion with banana-smell and rightward 
motion with fennel-smell, subjects exhibited more frequent perceptions of rightward motion in fennel-smell trials relative to banana-smell trials. 
(B): When pairing leftward motion with fennel-smell and rightward motion with banana-smell, subjects exhibited more frequent perceptions of 
rightward motion in banana-smell trials relative to fennel-smell trials. (C): When pairing both leftward and rightward motion with the same neutral air- 
smell the perceived motion directions were not significantly different across conditions. (D): Similar olfactory modulations on visual motion perception 
when the pairing relationships were randomized. Note only ambiguous rightward motion was used during the testing phase. 
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perceptions of ambiguous visual motion directions in a way that is 
consistent with the established odor-motion direction pairing. 

Experiment 2: olfactory modulation on visual perception was 
contingent on the odor-motion direction pairing. To confirm 
that the olfactory modulation on visual perception was specific to 
the associated pairing between odor-motion direction instead of a 
general perceptual and/or choice bias that was incidentally induced 
by the presence of smells, we conducted experiment 2 with the 
second group of subjects. This experiment followed the same 
procedures as experiment 1 except that we reversed the smell- 
motion direction pairing (Fig. 2B). When leftward motion 
direction was paired with smell of fennel and rightward motion 
direction with smell of banana, we found that modulation effects 
were reversed accordingly. Again, repeated measures ANOVA 
found significant effects of odorants (F(l,36) = 14.1, p < 0.001) 
and motion directions (F(l,36) = 127.6, p < 0.001) but no 
significant interactions either (F(l,36) = 0.39, p = 0.54). 
Specifically, with the reversed pairing, subjects perceived rightward 
motion more often in the banana-smell trials relative to the fennel- 
smell trials (t (9) = 3.1, p = 0.013 for leftward motion signals; and t 
(9) = 3.0, p = 0.015 for rightward motion signals). This result 
suggests that the observed modulations on ambiguous visual 
motion direction perceptions by olfactory cues were contingent 
upon the pre-established odor-motion direction association. 

Experiment 3: control for potential unidentified non-olfactory 
cues. One possibility is that the biases in direction perception 
between odors might reflect some artificial un-identified non- 
olfactory cues in our design. For example, any auditory, tactile, or 
visual cues might be interfering direction perception and causes the 
perceptual bias. To address this concern, we recruited an third 
independent group of subjects and asked them to perform the 
same task, except that both leftward and rightward motion 
directions were paired with the same neutral smell of air (from 
purified water). The subjects were instructed in a similar way as in 
the previous experiments. As one can see, in figure 2C, for both 
leftward and rightward ambiguous testing motion signals, there 
were no significant differences in direction perception between the 
two neutral smell conditions (t (9) = 0.03, p = 0.97 for leftward 
motion; and t (9) = 0.59, p = 0.57 for rightward motion). 
Correspondingly, the two-way ANOVA revealed no significant 
modulations (F(l,36) = 0.14, p = 0.72) and interactions (F(l,36) 
= 0.11, p = 0.75) either, but a significant effect on motion directions 
remained (F(l,36) = 110.1, p < 0.001). 

Experiment 4: control for randomization of odorant-motion 
direction pairing. In previous experiments, the fixed pairing 
stimuli of smell and visual motion direction were introduced in 
each trial. This procedure might potentially induce response or 
decisional bias into participants' direction judgments. To rule out 
this possibility, we introduced randomized odorant-motion 
direction pairings on a trial-by-trial basis. During the testing 
phase, only the ambiguous rightward motion (paired with either 
banana-like or fennel-like smells) was presented to subjects so that 
the total number of trials was equal to those in the previous 
experiments. This design gave rise to 4 total task conditions (2 
odorants X 2 pairings). With repeated measures ANOVA treating 
odorants and pairing as two independent factors, we found 
significant interactions (F(l,24) = 8.7, p = 0.007) between 
odorants and pairings, but no significant effects on odorants 
(F(l,24) = 0.04, p = 0.84) and pairing relationships (F(l,24) = 
0.19, p = 0.66). This means olfactory modulation on visual motion 
perception followed a very specific pattern: the presence of a smell 
(either banana- or fennel-like) biased participants towards rightward 
motion perception if it was paired with a rightward motion via 



associative conditioning, and vice versa for leftward motion 
(Fig. 2D). 

Discussion 

The current study demonstrates that olfactory cues can influence 
ambiguous visual motion direction discrimination. This finding adds 
to the sparse evidence for the cross-modal interactions that weak 
senses, such as olfaction, are capable of modulating dominant visual 
perception. Our result of olfaction- visual motion interactions com- 
plements recent studies of olfaction-visual object perception inter- 
plays 11,12 . These results indicate that the olfactory information 
processing and the visual information processing are functionally 
linked. Olfaction is more likely interacting with vision at multiple 
pathways, at least including ventral stream related to object recog- 
nition and dorsal stream related to motion representation. 

According to the Bayesian framework for multisensory integ- 
ration, the unified percept is the result of weighted estimates across 
modalities 2123 . Since the weights are proportional to the reliability in 
each modality and in primates visual perception generally prevail 
olfactory perception, previous studies on vision-olfaction interplays 
reported mainly the visual modulation on olfactory perception while 
the reversal observations are relatively rare (for review, see 10 ). In 
present study, we revealed this modulation effects by using ambigu- 
ous visual motion signals as probe to test the olfactory influences on 
visual motion processing. By manipulating the ambiguity levels of 
visual motion direction, we compromised the quality of visual signals 
to a large extent and therefore made the reliability comparable across 
senses, which is crucial in our experimental design. This similar 
concept of reducing visual reliability was largely employed in pre- 
vious cross-modal studies using the binocular rivalry para- 
digms 12,13,27,28 , in which degraded visual perception arises because 
of alternating (bistable) perceptions 29 . Taken together, these experi- 
ments including our current study, point to a general rule that redu- 
cing the reliability of dominant sense facilitates our appreciations of 
cross-modal interactions. 

In our task design, we used visual motion signals instead of pre- 
vious widely used visual color/shapes or pictorial images 3-8,11,12 . In 
odor detection/discrimination tasks subjects showed enhanced per- 
formance when an odor was presented together with a congruent 
color/picture in comparison to a non-matching one 7 . On the other 
hand, in visual object perception tasks, the presence of a semantically 
congruent odor facilitated visual object perceptions 11,12 , as compar- 
ing to the case of the non-congruent odor. All these previous studies 
used the visual attributes processed by the visual ventral pathway. 
Therefore, interplays between olfaction and vision in these studies 
might indicate a functional merging between the visual ventral 
stream and the olfactory system. Here, in the current study, with 
the olfactory modulation on the visual motion signals, we extend 
the current understandings of cross-modal interactions to the visual 
dorsal pathway. We should point out that such changes in visual 
appearance by means of Pavlovian conditioning have been already 
demonstrated previously 30 ' 32 . For example, Haijiang et al (2006) 
described a "cue recruitment" experiment in which an association 
between an unrelated cue and the rotating direction of a Necker cube 
was established during the training phase. Subsequently, on test trials 
the presence of this unrelated cue became effective in biasing parti- 
cipants' direction judgment of the ambiguous rotating cube. In the 
current study we could as well have employed similar bi-stable 
motion signals to pair with olfactory cues. However, our point here 
was not to replicate Haijiang et al's findings but rather was attempted 
to test whether olfactory cues could influence visual motion proces- 
sing in general. Similar effects have also been observed in audiovisual 
interaction studies 33 35 . It was reported that the association between 
sound and visual motion was easily acquired within a short period of 
time and sound was able to induce illusory visual motion perception 
for both static 33 and moving visual objects 34,35 . These data and ours 
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suggest that the visual system can flexibly utilize new signals (e.g., 
auditory or olfactory) as "cues" to build perceptions after exposure to 
correlated input signals via the form of multisensory association,- 
which can develop very rapidly 3637 . 

We would like to point out that the results of current study are 
based on behavioral experiments. Therefore, we can only say there is 
a direct/indirect functional link exists between dorsal visual pathway 
and olfactory system, but we are not sure about the underlying neural 
mechanism yet. The linkages between these two sensory systems 
could in principle have taken place either in the earlier/lower areas 
or in the later/higher cortical areas. In line with the early merging 
hypothesis, olfaction-vision interaction has shown to be subcon- 
scious 12 and nostril-visual field specific 13 . Furthermore, TMS stimu- 
lation over human VI improves olfactory discrimination 38 , and 
olfactory stimulation selectively modulates the OFF bipolar cells in 
the retina of zebra fish 39 . Alternatively, the cross-talk between these 
two senses could also be a result of top-down projections from higher 
cortical areas such as the orbitofrontal cortex 40 , which receives con- 
verging input from both the primary olfactory cortex and higher- 
order visual areas 41 and has been suggested to play important roles in 
integrating olfactory and visual information 7 ' 42,43 . A recent neuro- 
physiology study 44 revealed that neurons in a motion-sensitive cor- 
tical area named middle temporal (MT) became selective for the 
orientations of static arrows after monkeys had learned the asso- 
ciation between arrow orientations and different motion directions 
- a property not seen prior to the associative learning. In the current 
study the associations between the odorants and motion directions 
were arbitrarily established by means of classical Pavlovian condi- 
tioning. Obviously, there was a lacking of natural "congruency" 
between the directions of moving dots and the smells of odorants. 
We thus consider it unlikely that a lower level interaction across these 
two sensory systems had occurred. Instead, we are inclined to specu- 
late that the interplay between the olfactory system and the visual 
dorsal stream is more likely to be operating at the semantic/cognitive 
level that originates from higher cognitive brain areas 40 ' 43 . However, 
we admit that without further evidence we cannot rule out the roles 
of early cortical processing. For example, in audiovisual interaction, 
studies have suggested relatively low level perceptual processing 
could be involved in the paired association of arbitrary pitched sound 
and visual motion within a short period 45,46 . Be it at high levels or not, 
the flexible cross-modal interactions reported in this study via short- 
term associative learning are consistent with the general view of a 
highly plastic brain. This extraordinary plasticity during multisen- 
sory integration may provide implications into the field of sensory 
substitution studies 47,48 . 

In summary, the present study has shown that, after establishing 
an association between odorants and motion directions, olfactory 
cues can bias perceptions of ambiguous visual motion directions. 
These results extend our understandings on vision-olfaction cou- 
plings and suggest a functional interaction between the olfactory 
system and the visual dorsal pathway. Further investigations invol- 
ving brain imaging or extracellular recording techniques are needed 
to address precisely at which neural level the interaction takes place 
and what the underlying neuronal mechanism are. 

Methods 

Subjects. A total of forty-one subjects (21 females and 20 males, mean age ± SEM = 
24.0 ± 0.79 years) volunteered to participate in the experiments. We randomly 
divided them into four groups, with each group participated in one of the four 
experimental designs. All subjects have reported to have normal olfaction and normal 
or corrected -to -normal vision. They were naive with respect to the scientific purpose 
of this study. All experimental protocols were approved by the Ethics Review 
Committee of Institute of Psychology, CAS. All the experimental procedures 
described below were conducted in accordance with institutional ethical guidelines 
and informed consent was obtained from all subjects. 

Apparatus and stimuli. Subjects were seated in a dimly lit room. They fixated their 
heads with a chinrest, which was 57 cm away from the CRT monitor (refresh rate: 
60 Hz). The generation of visual stimuli and experimental tasks were designed in 



Matlab, using the Psychophysics Toolbox extensions 49 ' 50 . For visual fixation spot, a 
gray circle (diameter: 0.25deg) was presented in the center of the screen on a black 
background. Moving dot patches consisted of 100 dots (diameter: 0.05deg) that were 
randomly located within an invisible circular window (diameter: 7.5deg). The moving 
dot patch was presented above the central fixation spot with an eccentricity of 7.5deg. 
There were two kinds of moving dot patches. In the coherent dot patch, all the dots 
were moving in the same direction (either left or right, at a speed of 3 deg/s). In the 
ambiguous dot patch, only a fraction of dots were moving in the same direction while 
others were moving randomly. Varying the fractions of randomly moving dots allows 
us to manipulate the ambiguity levels of motion directions. For each subject we pre- 
tested their direction discrimination performance as a function of fractions of 
random dots to obtain a psychometric curve. The fraction that gave rise to 
approximately 60% correct performance was then used for the ambiguous moving 
dot patch in the subsequent main experiment. We used only one coherence level 
instead of the full range (say, from 0% to 100%) as typically was done to obtain a 
psychometric function. Here we were constrained by relative longer inter-trial 
intervals with olfactory stimuli, and more critically, olfactory processing is vague and 
fuzzy and it may saturate (or lose sensitivity) after a limited number of trials with 
repeated exposure to odorants (see below).We chose the ambiguity level of 60% 
performance based on the empirical experience which was constrained by the fact 
that, on the one hand, the reliability of visual motion signals should be sufficiently low 
(so as to be comparable with olfactory cues), and on the other hand, we need to at least 
reserve some directional information for subjects in the motion direction 
discrimination task. 

We used similar procedures as described in the previous study 13 to generate 
olfactory stimuli and deliver them to participants, except that in the current study the 
olfactory stimuli delivered to both nostrils were identical. In brief, we had two kinds of 
odorants: banana-like odor (isoamyl acetate, 1% in propylene glycol) and fennel-like 
odor (anethole, 1% in propylene glycol). They were contained in two separate 40 ml 
glass bottles, each fitted with a Teflon nosepiece. Each odorant was delivered to both 
nostrils of the participants using a computer- controlled air-dilution olfactometer. 
Participants were encouraged to inhale with their nose and exhale through their 
mouth. All olfactory stimuli were suprathreshold to all subjects, and this was validated 
by post-experiment oral reports. 

Task design. The behavioral task was the same for subjects in each experimental 
condition: they were required to indicate the perceived direction of the moving dot 
patch by means of keyboard press (left key for leftward motion and right key for 
rightward motion). 

Each trial was consisted of two successive phases: a pairing phase and a testing 
phase (Fig. 1). Each trial started when subjects fixed their gaze at the central gray 
fixation spot for 1 s, followed by a coherently moving dot patch appeared in the visual 
periphery for 3 s during which subjects were instructed (the fixation spot transiently 
turn red) to inhale a specific smell (e.g., banana-like). After a brief inter- stimulus - 
interval (ISI), a coherent moving dot patch with opposite moving direction would 
appear at the same location for another 3 s and subjects were asked to inhale a 
different smell (fennel-like). Note that the order in which the smell-motion direction 
pair appeared was randomly interleaved during the pairing phase. The subsequent 
testing phase began after a second ISI of 1.5 s. During the testing phase, the 
ambiguous moving dot patch would appear and subjects were requested to inhale for 
a third time before they gave a keyboard response to indicate the perceived direction. 
The directions of ambiguous moving dot patch could be either left or right and the 
accompanying odorants could be either banana or fennel smell, and this gave rise to 4 
total task conditions (2 directions X 2 odorants). The motion directions and odorants 
were balanced out and randomly interleaved from trial to trial. There is a 30 s break 
between each trial with continuous exposure to air puffs to flush the olfactory 
memory. Each subject performed a total of 160 trials (120 in the pre-test and 40 in the 
subsequent experiment). Since there were two directions, if the participants made 
their responses or decisions based not on the testing visual motion but on the pre- 
viewed pairing of the smell and the motion, then we should expect no difference 
between left and right testing stimuli. In fact this turned out not to be the case in our 
data (see Results). 

In experiment 1 , we paired the banana smell with leftward moving coherent dot 
patch (while the fennel smell with rightward moving coherent dot patch). To verify 
our observations in experiment 1, in experiment 2 we reversed the odor- direction 
pair. That is, we paired the banana smell with rightward moving coherent dot patch. 
As a control experiment to rule out any potential unidentified non-olfactory cues that 
contributed to our results, in experiment 3 we paired both directions with the same 
neutral smell from the purified water which was otherwise suggested to the partici- 
pants as containing different smells with low intensity. In experiment 4, we inter- 
mixed the pairing relationships (motion-odorant mappings were randomly 
interleaved from trial to trial) to discourage any response and/or decision biases. 

Data analysis. In experiment 1-3, our task designs involved 2 (motion directions) X 
2 (odorants) conditions. In each condition we counted the percentages of the 
rightward motion perceptions for each subject. These percentages were then averaged 
across subjects in each condition. We used both a two-way repeated measures 
ANOVA and a post-hoc paired t-test to statistically test the modulations of olfactory 
input on motion perception. P values less than 0.05 were considered statistically 
significant and marked with one asterisk, and less than 0.01 was marked with two 
asterisks. In experiment 4 the two-factor repeated measures included 2 odorants and 
2 pairing relationships (while the direction was fixed as the ambiguous rightward 
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moving patch). Otherwise the analysis and statistical tests were identical as those in 
previous experiments. 
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